Graphical user interfaces have replaced text-only interfaces in the vast majority of today's computer environments. The typical graphical user interface conveys at least some of the information to users via non-text images. Items may be selected or activated through the use of a point-and-click device, such as a mouse or trackball. It is important to note that point-and-click targets may occur anywhere on the screen.
The most popular graphically oriented operating systems are the windows-based offerings of Microsoft Corporation, with Windows 95™, Windows 98™, Windows for Workgroups™, Windows NT™, Windows 2000™, Windows 2003™, Windows ME ™, and Windows XP™ being examples. A “window” is an area of the display set aside for a specific purpose. Within each window, there will often be graphical images, as well as many different locations that can be targets for the point-and-click devices.
One feature of the typical modern computer is that it is able to perform many tasks simultaneously. The modern graphical user interface reflects this capability by allowing many different windows to be “open” simultaneously, with each window corresponding to a different piece of software, or application program. Thus, different application programs can be displayed simultaneously in different windows. The window commonly has visible boundaries within which information is displayed by the corresponding application program. The active window is the window with which the user is currently interacting while an inactive window is an open, but not currently active, window. (To illustrate the difference, although visible changes can occur in any window that is open—i.e., any window that is being displayed on the screen—the active window is the one in which users' keyboard entries will have an effect. If windows overlap, the active window will appear to be the one on top, concealing portions of the other windows in locations where they overlap.) The user can control the size, shape, and positioning of the windows by appropriate positioning of the cursor and keyboard/mouse commands.
Because windows can overlap and “hide” one another, a display tool bar is provided in windows-based operating systems. This tool bar may be positioned by users along any edge of the screen (most prefer the bottom edge); it contains icons that represent applications that may be invoked, as well as individual icons that correspond to each of the open windows.
As the number of windows opened becomes large, the display tool bar can become increasingly crowded with icons and unhelpful to the user. To make matters worse, the ordering of the window icons in the tool bar do not reflect the precise ordering of the window stack. The icons are ordered based upon the relative timing of when the windows were opened and not based upon the relative positions of the windows on the display. This forces the user to periodically click on the windows to identify which windows are open and rearrange them so that a part of each opened window is visible. Additionally, windows-based operating systems normally require two hands and full attention to operate. The user must not only view the display but also make selections or enter data via a point-and-click device, e.g., a mouse, a track ball, joy stick, stylus, etc. In many implementations, the user does not have the convenience of providing both hands and complete attention to the computer, thereby diminishing the attractiveness and decreasing the productivity of the operating system. For example, when operating a vehicle, it can be unsafe for a driver to view and operate a device that utilizes a visual display. There are situations, such as night-time use of the device, where illuminated displays are undesirable due to the adverse impact on the user's night vision. Similarly, the use of an illuminated display by military personnel at night may be undesirable because the illumination can reveal the user's location.
Another consideration is that disability access is increasingly becoming an important issue to software and hardware manufacturers. Many countries are enacting stringent laws requiring products to have specific features that permit the products to be operated by people with disabilities. An illustrative example is Section 508 of the Rehabilitation Act of 1998 (29 U.S.C. §794d). The requirements associated with this legislation, that apply specifically to computer-based graphical user interfaces, are in the Code of Federal Regulations, 36 CFR Parts 1194.21 and 1194.22. The majority of these requirements are oriented toward people who unable to see. For example, regulation 1194.21(a) requires all operations to be accessible via the computer keyboard—i.e., without a point-and-click device. This regulation is necessary because people who are blind are unable to discern the position of the on-screen pointer, the locations of targets within windows, and the relative positions and borders of the different windows that may be open.
In order to operate graphical user interfaces, people who are blind rely on sophisticated text-to-speech assistive software applications, such as Microsoft Narrator™, JAWS™ by Freedom Scientific BLV Group, LLC, and Window-Eyes™ by GW Micro. Navigation within and between windows is accomplished by using the TAB, ALT, and arrow keys, either singly or in combination with other keys. For example, pressing TAB within a typical window causes the programmatic focus to move to the next tab-stop or point-and-click target. With each press of the TAB key, the text-to-speech software provides a spoken description of the target. (Section 508 regulation 1194.22(a) requires all non-text elements to have an underlying text tag. This is what permits the text-to-speech software to respond appropriately when the TAB function takes the focus to an icon or other graphical image.) When the desired target is voiced-out, the function may be executed by pressing the ENTER key.
A similar mechanism allows users to move among different applications, and to select the window to be activated. This is accomplished by pressing the ALT key continuously while pressing and releasing the TAB key. With each press of the TAB key, users may move sequentially through the windows that have been opened. The sequence of movement is in chronological order, starting with the window that had been activated most recently. (Note that the order in which the icons appear in the tool bar is based on the sequence in which the corresponding windows were first opened, not the sequence in which the windows had been activated.) Immediately after each release of the TAB key (but not the ALT key), the assistive text-to-speech software will read the title bar of the selected window. When the user hears the software speak the title of the desired window, the user causes that window to be activated by releasing the ALT key.
Although support for keyboard-based control does permit graphical user interfaces to be operated by people who are blind, this style of operation is considerably slower and more tedious than what is available to people who are able to operate point-and-click devices. To reiterate a point made earlier, the fundamental problem is that blind users are unable to discern the position of the on-screen pointer, the locations of targets within windows, and the relative positions and borders of the different windows that may be open. In this context, it is important to note that sighted users often form a mental map of where desirable point-and-click targets are located, thereby permitting them to move the pointer to the target very quickly. The lack of memorable location-specific data in the information presented to blind users represents one more operational advantage that is presently available only to sighted users.